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- The MAILING DATE of this communication appears on the cover sheet with the correspondence address - 
Period for Reply 

A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1.136(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 
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earned patent term adjustment. See 37 CFR 1.704(b). 

Status 

1)13 Responsive to connmunication(s) filed on 73 November 2002 . 
2a)n This action is FINAL. 2b)H This action is non-final. 

3) n Since this application is in condition for allowance except for fonnal matters, prosecution as to the merits is 

closed in accordance with the practice under Expa/te Quay/e, 1935 CD. 11, 453 O.G. 213. 
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4a) Of the above claim(s) is/are withdrawn from consideration. 

5) 0 Claim(s) is/are allowed. 
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If approved, corrected drawings are required in reply to this Office action. 

12) 0 The oath or declaration is objected to by the Examiner. 
Priority under 35 U.S.C. §§119 and 120 

13) H Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 1 19(a)-(d) or (f). 

a)l3AII b)n Some*c)n None of: 
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2. n Certified copies of the priority documents have been received in Application No. . 

3. n Copies of the certified copies of the priority documents have been received in this National Stage 
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DETAILED ACTION 



1 . This action is responsive to communications: Pre-Amendment, filed on 
11/13/2002. This action is non-final. 

2. Claims 1, 3-4, 10, 31-39 are pending in this application. Claims 1, 10 and 31 are 
independent claims. In the Pre-Amendment, filed on 11/13/2002, claims 1, 3-4, 10 and 
31 were amended, claims 2, 5-9 and 1 1-30 were canceled, and claims 32-39 were 
added. 

This application claims foreign priority dated 3/26/1999. 

3. The present title of the invention is "Image processing method and apparatus, 
and storage medium". 



4. The following is a quotation of 35 U.S.C. 1 03(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 1 02 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 



5. Claims 1, 3-4, 10, 31-32 and 35-37 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Chen et aL (5,613,048), admitted prior art and Hrytzak et al. 



Claim Rejections - 35 USC § 103 



(5,327,257). 
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6. As per claim 1 , Chen et al., hereinafter Chen, an image processing apparatus for 
generating an image of a virtual space according to a user's operating using ray space 
data comprising: 

a recording unit adapted to record the ray space data ("selected views of a 
scene, from different viewpoints, are recorded", column 3, line 53-54); 

a setting unit adapted to set a sampling rate for ray space data read out from 
said recording unit according to the user's operation, the sampling rate indicating a 
distance between pixels to be sampled ("interpolation is carried out through image 
morphing. Generally speaking, image morphing is a simultaneous interpolation of shape 
and texture. The morphing technique involves two basic steps. In the first step, a 
correspondence is established between two images ... Typically, the correspondence is 
established by a human operator ... An algorithm is then employed to determine the 
correspondence, or mapping, of the remaining points of the images", column 4, line 1- 
14, "The sampling rate is determined by the largest offset vector from the morph map", 
column 8, line 26-28, since the morph map is established by a human operator, the 
sampling rate is indirectly set by a human operator, and "a foreground object in a scene 
should be sampled at a high rate, while a stationary wall in the background needs only a 
few samples", column 8, line 35-37, this indicate the sampling rate is a function of 
distance between pixels to be sampled); 

a reconstructed unit adapted to read out ray space data from said recording unit 
according to the sampling rate set by said setting unit, and to reconstruct an image of 
the virtual space ("Once each of these views has been stored, the image from any 
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viewpoint between any two of the recorded locations can be obtained by interpolation of 
two adjacent stored images", column 3, line 60-4). 

Chen discloses an apparatus of generating an image of a virtual space. It is 
noted that Chen does not explicitly disclose using ray space theory represent an image, 
however, this is known in the admitted prior art. In the admitted prior art, applicant 
disclosed a ray space theory has been used to describe a virtual space (page 1 , line 19- 
20). 

Thus, it would have been obvious to one of ordinary skill in the art at the time the 
invention was made to incorporate the teaching of admitted prior art into the invention in 
order to properly describe an image with a moving view point. 

It is further noted that Chen and the admitted prior art do not explicitly disclose an 
interpolation unit adapted to interpolated pixels of the image reconstructed by said 
reconstructing unit until the size of the image becomes a predetermined size, however, 
this is known in the art as taught by Hrytzak et al., hereinafter Hrytzak. Hrytzak 
discloses a method of interpolating a digital image in which "the method can be 
repeated by moving to the next local position in the input image and so on until the 
entire image has been interpolated, i.e. until a complete output image of the required 
size is produced", column 2, line 7-1 1 . 

Thus, it would have been obvious to one of ordinary skill in the art at the time the 
invention was made to incorporate the teaching of Hrytzak into Chen and admitted prior 
art because Chen and admitted prior disclose an apparatus of generating an image of a 



Application/Control Number: 09/500,455 Page 5 

Art Unit: 2672 

virtual space using ray space theory and Hrytzak discloses the generated innage can be 
interpolated into a desired size in order to maxinnized the view of the interpolated innage. 

7. As per clainn 3, Chen, adnnitted prior art and Hrytzak dennonstrated all the 
elennents as applied to the rejection of independent clainn 1 , supra, and Chen further 
discloses the setting unit determines the sampling rate on the basis of the moving 
speed in a virtual space designated by the user ("The movement of the virtual camera, 
to present different views of the scene, is controlled by the viewer through a suitable 
input device 18", column 3, line 40-42, since the morphing, thus the sampling rate, is 
determined by the correspondence between two images and correspondence depends 
on the moving speed of the camera). 

8. As per claim 4, Chen, admitted prior art and Hrytzak demonstrated all the 
elements as applied to the rejection of independent claim 1 , supra, and Chen further 
discloses the setting unit determines the sampling rate on the basis of the manipulation 
speed of an object in a virtual space designated by the user ("The movement of the 
virtual camera, to present different views of the scene, is controlled by the viewer 
through a suitable input device 18, column 3, line 40-42, since the morphing, thus the 
sampling rate, is determined by the correspondence between two images and the 
correspondence depends on the moving speed of the camera). 

9. As per claim 10, Chen discloses an image processing method for reconstructing 
an image of a virtual space according to a user's operating using ray space data 
comprising: 
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a setting step of setting a sampling rate for ray space data read out from said 
recording means according to the user's operation, the sampling rate indicating a 
distance between pixels to be sample ("interpolation is carried out through image 
morphing. Generally speaking, image morphing is a simultaneous interpolation of shape 
and texture. The morphing technique involves two basic steps. In the first step, a 
correspondence is established between two images ... Typically, the correspondence is 
established by a human operator ... An algorithm is then employed to determine the 
correspondence, or mapping, of the remaining points of the images", column 4, line 1- 
14, "The sampling rate is determined by the largest offset vector from the morph map", 
column 8, line 26-28, since the morph map is established by a human operator, the 
sampling rate is indirectly set by a human operator, and "a foreground object in a scene 
should be sampled at a high rate, while a stationary wall in the background needs only a 
few samples", column 8, line 35-37, this indicate the sampling rate is a function of 
distance between pixels to be sampled); 

a reconstruction step of reading out ray space data from the recording means in 
accordance with the sampling rate set in said setting step, and reconstructing an image 
of the virtual space ("Once each of these views has been stored, the image from any 
viewpoint between any two of the recorded locations can be obtained by interpolation of 
two adjacent stored images", column 3, line 60-4). 

Chen discloses a method of generating an image of a virtual space. It is noted 
that Chen does not explicitly disclose using ray space theory represent an image, 
however, this is known in the admitted prior art. In the admitted prior art, applicant 
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disclosed a ray space theory has been used to describe a virtual space (page 1 , line 19- 
20). 

Thus, it would have been obvious to one of ordinary skill in the art at the time the 
invention was made to incorporate the teaching of admitted prior art into the invention in 
order to properly describe an image with a moving view point. 

It is further noted that Chen and the admitted prior art do not explicitly disclose an 
interpolation step of interpolating pixels of the image reconstructed by said 
reconstructing unit until the size of the image becomes a predetermined size, however, 
this is known in the art as taught by Hrytzak et al., hereinafter Hrytzak. Hrytzak 
discloses a method of interpolating a digital image in which "the method can be 
repeated by moving to the next local position in the input image and so on until the 
entire image has been interpolated, i.e. until a complete output image of the required 
size is produced", column 2, line 7-1 1 . 

Thus, it would have been obvious to one of ordinary skill in the art at the time the 
invention was made to incorporate the teaching of Hrytzak into Chen and admitted prior 
art because Chen and admitted prior disclose a method of generating an image of a 
virtual space using ray space theory and Hrytzak discloses the generated image can be 
interpolated into a desired size in order to maximized the view of the interpolated image. 
10. As per claim 31 , Chen discloses a computer-readable storage medium for storing 
a program which makes a computer function as an image processing apparatus for 
generating an image virtual space according to a user's operating using ray space data 
recorded in a memory, the program comprising: 
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a setting step of setting a sannpling rate for ray space data read out from said 
recording means according to the user's operation, the sampling rate indicating a 
distance between pixels to be sample ("interpolation is carried out through image 
morphing. Generally speaking, image morphing is a simultaneous interpolation of shape 
and texture. The morphing technique involves two basic steps. In the first step, a 
correspondence is established between two images ... Typically, the correspondence is 
established by a human operator ... An algorithm is then employed to determine the 
correspondence, or mapping, of the remaining points of the images", column 4, line 1- 
14, "The sampling rate is determined by the largest offset vector from the morph map", 
column 8, line 26-28, since the morph map is established by a human operator, the 
sampling rate is indirectly set by a human operator, and "a foreground object in a scene 
should be sampled at a high rate, while a stationary wall in the background needs only a 
few samples", column 8, line 35-37, this indicate the sampling rate is a function of 
distance between pixels to be sampled); 

a reconstruction step of reading out ray space data from the memory according 
to the sampling rate set in said setting step, and reconstructing an image of the virtual 
space ("Once each of these views has been stored, the image from any viewpoint 
between any two of the recorded locations can be obtained by interpolation of two 
adjacent stored images", column 3, line 60-4). 

Chen discloses a method of generating an image of a virtual space. It is noted 
that Chen does not explicitly disclose using ray space theory represent an image, 
however, this is known in the admitted prior art. In the admitted prior art, applicant 
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disclosed a ray space theory has been used to describe a virtual space (page 1, line 19- 
20). 

Thus, it would have been obvious to one of ordinary skill in the art at the time the 
invention was made to incorporate the teaching of admitted prior art into the invention in 
order to properly describe an image with a moving view point. 

It is further noted that Chen and the admitted prior art do not explicitly disclose an 
interpolation step of interpolating pixels of the image reconstructed by said 
reconstructing unit until the size of the image becomes a predetermined size, however, 
this is known in the art as taught by Hrytzak et al., hereinafter Hrytzak. Hrytzak 
discloses a method of interpolating a digital image in which "the method can be 
repeated by moving to the next local position in the input image and so on until the 
entire image has been interpolated, i.e. until a complete output image of the required 
size is produced", column 2, line 7-1 1 . 

Thus, it would have been obvious to one of ordinary skill in the art at the time the 
invention was made to incorporate the teaching of Hrytzak into Chen and admitted prior 
art because Chen and admitted prior disclose a method of generating an image of a 
virtual space using ray space theory and Hrytzak discloses the generated image can be 
interpolated into a desired size in order to maximized the view of the interpolated image. 
11. As per claim 32, Chen, admitted prior art and Hrytzak demonstrated all the 
elements as applied to the rejection of independent claim 1 , supra, and admitted prior 
art further discloses wherein ray space data is managed in a (x,u) space, wherein "x" 
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represents a position where he light ray intersects the X-axis and "u" represents a 
tangent of an angle the light ray make with the Z-axis (page 2, line 19-23). 

Thus, it would have been obvious to one of ordinary skill in the art at the time the 
invention was made to incorporate the teaching of admitted prior art into the invention in 
order to properly describe an image with a moving view point. 

12. As per claim 35, Chen, admitted prior art and Hrytzak demonstrated all the 
elements as applied to the rejection of independent claim 1, supra, and Chen further 
discloses the setting unit determines the sampling rate on the basis of the manipulation 
speed of an object in a virtual space designated by the user ("The movement of the 
virtual camera, to present different views of the scene, is controlled by the viewer 
through a suitable input device 18, column 3, line 40-42, since the morphing, thus the 
sampling rate, is determined by the correspondence between two images and the 
correspondence depends on the moving speed of the camera). 

13. As per claim 36, Chen, admitted prior art and Hrytzak demonstrated all the 
elements as applied to the rejection of independent claim 1, supra, and Chen further 
discloses the setting unit determines the sampling rate on the basis of the manipulation 
speed of an object in a virtual space designated by the user ("The movement of the 
virtual camera, to present different views of the scene, is controlled by the viewer 
through a suitable input device 18, column 3, line 40-42, since the morphing, thus the 
sampling rate, is determined by the correspondence between two images and the 
correspondence depends on the moving speed of the camera). 
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14. As per claim 37, Chen, admitted prior art and Hrytzak demonstrated all the 
elements as applied to the rejection of independent claim 10, supra, and admitted prior 
art further discloses wherein ray space data is managed in a (x,u) space, wherein "x" 
represents a position where he light ray intersects the X-axis and "u" represents a 
tangent of an angle the light ray make with the Z-axis (page 2, line 1 9-23). 

Thus, it would have been obvious to one of ordinary skill in the art at the time the 
invention was made to incorporate the teaching of admitted prior art into the invention in 
order to properly describe an image with a moving view point. 

Allowable Subject Matter 

15. Claims 33-34 and 38-39 are objected to as being dependent upon a rejected 
base claim, but would be allowable if rewritten in independent form including all of the 
limitations of the base claim and any intervening claims. 

Conclusion 

16. The prior art made of record and not relied upon is considered pertinent to 
applicant's disclosure. 



Inquiries 
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17. Any inquiry concerning this comnnuni cation or earlier communications from the 
examiner should be directed to Ryan Yang whose telephone number is (703) 308- 
6133. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Michael Razavi, can be reached at (703) 305-4713. 

Any response to this action should be mailed to: 



or faxed to: 

(703) 872-9314 (for Technology Center 2600 only) 

Hand-delivered responses should be brought to Crystal Park II, 2121 Crystal 
Drive, Arlington, VA, Sixth Floor (Receptionist). 

Any inquiry of a general nature or relating to the status of this application or 
proceeding should be directed to the Technology Center 2600 Customer Service Office 
whose telephone number is (703) 306-0377. 



Commissioner of Patents and Trademarks 



Washington, D.C. 20231 




Ryan Yang 
September 1 1 , 2003 



